Head mounted display, display control device, processing method, display method, and storage medium

ABSTRACT

A non-transitory computer-readable storage medium storing a program that causes a processor of a head mounted display to execute a process, the process includes obtaining a result of measurement performed by an acceleration sensor included in the head mounted display; and determining whether a user with the head mounted display is chewing a food or the user is speaking based on the result of the measurement performed by the acceleration sensor.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of theprior Japanese Patent Application No. 2017-232771, filed on Dec. 4,2017, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to a head mounted display,a display control device, a processing method, a display method, and astorage medium.

BACKGROUND

In recent years, a head mounted display (HMD) has been developed andcommercialized. Recently, a technique of supporting a user on arestricted diet by displaying calories and ingredients of foods in frontof the user with the HMD or a technique of having a meal while the uservirtually faces another remote user by displaying the remote user in theHMD have been studied.

In the HMD, a technique of detecting eyestrain of a user and notifyingthe user of the eyestrain has been used (refer to Japanese Laid-openPatent Publication No. 2004-286832, for example).

However, in Japanese Laid-open Patent Publication No. 2004-286832,usability of the HMD during a meal has not been discussed. Accordingly,the present disclosure preferably contributes to improvement ofusability of the head mounted display during a meal.

SUMMARY

According to an aspect of the embodiments, a non-transitorycomputer-readable storage medium storing a program that causes aprocessor of a head mounted display to execute a process, the processincludes obtaining a result of measurement performed by an accelerationsensor included in the head mounted display; and determining whether auser with the head mounted display is chewing a food or the user isspeaking based on the result of the measurement performed by theacceleration sensor.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram schematically illustrating a configuration of a headmounted display according to a first embodiment;

FIG. 2 is a diagram illustrating a hardware configuration of a controldevice of FIG. 1;

FIG. 3 is a functional block diagram of the control device;

FIG. 4 is a flowchart of a process performed by the control device;

FIG. 5 is a flowchart of a detailed process performed in step S26 ofFIG. 4;

FIG. 6A is a graph illustrating a waveform of acceleration obtained whena user is chewing a food;

FIG. 6B is a graph illustrating a waveform of acceleration obtained whenthe user is speaking;

FIG. 7A is a graph illustrating a waveform of acceleration obtained whenthe user is chewing a hard food;

FIG. 7B is a graph illustrating a waveform of acceleration obtained whenthe user is chewing a soft food;

FIG. 8 is a functional block diagram of a control device according to asecond embodiment;

FIG. 9 is a flowchart of a process performed by the control deviceaccording to the second embodiment; and

FIG. 10 is a diagram illustrating a fatigue determination table.

DESCRIPTION OF EMBODIMENTS First Embodiment

Hereinafter, a head mounted display according to a first embodiment willbe described in detail with reference to FIGS. 1 to 7B. In FIG. 1, ahardware configuration of a head mounted display (HMD) 100 according toa first embodiment is schematically illustrated.

The HMD 100 is an apparatus which may be worn by a user on a head.Examples of the HMD 100 include a monocular transparent type HMD. Theexamples of the HMD 100 further include various HMDs, such as abinocular HMD and an immersive HMD.

As illustrated in FIG. 1, the HMD 100 includes an acceleration sensor12, a gyroscope sensor 14, a camera 16 serving as an imaging unit, amicrophone 18, an input device 20, a display device 22 serving as adisplay unit, an audio output device 24, a communication device 26, anda control device 10 serving as an information processing device.

The acceleration sensor 12 detects acceleration of the HMD 100. Examplesof the acceleration sensor 12 include a piezoresistance triaxialaccelerometer and an electrostatic capacitance triaxial accelerometer. Aresult of measurement performed by the acceleration sensor 12 istransmitted to the control device 10.

The gyroscope sensor 14 detects an angle (an orientation), an angularvelocity, or angular acceleration of the HMD 100. Examples of thegyroscope sensor 14 include a vibration gyroscope sensor. A result ofmeasurement performed by the gyroscope sensor 14 is transmitted to thecontrol device 10. The gyroscope sensor 14 may detect an inclination ofa head of the user who wears the HMD 100, for example.

The HMD 100 may include, in addition to the acceleration sensor 12 andthe gyroscope sensor 14, a direction sensor, a geomagnetic sensor, and aglobal positioning system (GPS).

The camera 16 is disposed in the vicinity of eyes of the user andperforms imaging in a gaze direction of the user with the HMD 100. Animage captured by the camera 16 is transmitted to the control device 10.

The microphone 18 collects user voice and surrounding sound. The inputdevice 20 includes a touchpad and buttons, such as a power button, ahome button, and a volume control button.

The display device 22 displays various information. The display device22 projects information on a half mirror. Specifically, the displaydevice 22 is a transparent type display device which allows the user toview an outside scene in a transparent manner along with theinformation. The display device 22 may be an immersive display type, avideo transparent type, or a retinal projection type.

The audio output device 24, which is a speaker or earphones, outputsaudio information under control of the control device 10.

The communication device 26 receives a wireless signal or communicationdata supplied from a base station (a mobile network) through an antennaor the like and transmits a wireless signal or communication data to thebase station. The communication device 26 performs short rangecommunication with a computer, such as another terminal, using a shortrange communication method, such as infrared communication, WiFi, orBluetooth (registered trademark).

The control device 10 integrally controls the units included in the HMD100. In particular, the control device 10 has a function ofappropriately controlling display of the display device 22 while theuser with the HMD 100 is having a meal in this embodiment. FIG. 2 is adiagram illustrating a hardware configuration of the control device 10.As illustrated in FIG. 2, the control device 10 includes a centralprocessing unit (CPU) 90, a read only memory (ROM) 92, a random accessmemory (RAM) 94, a storage unit (a solid state drive (SSD) or a flashmemory in this embodiment) 96, an input/output interface 97, and aportable storage medium drive 99. The components included in the controldevice 10 are connected to a bus 98. The input/output interface 97 isconnected to the various devices other than the control device 10 ofFIG. 1. In the control device 10, programs (including a processingprogram and a display program) stored in the ROM 92 or the storage unit96 or programs (including a processing program and a display program)read by the portable storage medium drive 99 from a portable storagemedium 91 are executed by the CPU 90 so that functions of the unitsillustrated in FIG. 3 are realized. Each of the functions of the unitsof FIG. 3 may be realized by an integrated circuit, such as anapplication specific integrated circuit (ASIC) or a field programmablegate array (FPGA).

FIG. 3 is a functional block diagram of the control device 10. Asillustrated in FIG. 3, the control device 10 functions as anacceleration obtaining unit 50 serving as an obtaining unit, amastication determination unit 52 serving as a determination unit, afatigue degree calculation unit 54 serving as a calculation unit, and adisplay controller 56 when the CPU 90 executes the programs.

The acceleration obtaining unit 50 obtains a result of measurementperformed by the acceleration sensor 12 and transmits the result to themastication determination unit 52.

The mastication determination unit 52 determines whether a mouth of theuser is moving in accordance with the measurement result of theacceleration sensor 12. When the determination is affirmative, themastication determination unit 52 determines whether the user is chewing(masticating) a food (an ingredient) or the user is speaking.

When it is determined that the user is chewing a food, the fatiguedegree calculation unit 54 determines whether the user gets tired due tothe chewing. When the determination is affirmative, a fatigue point isincremented. In this embodiment, it is assumed that the fatigue degreecalculation unit 54 increments a fatigue point when the user chews ahard food and does not increment a fatigue point when the user chews asoft food since the user does not get tired.

The display controller 56 controls display in the display device 22based on accumulated fatigue points.

Processing of Control Device 10

Next, processing performed by the control device 10 will be described indetail with reference to FIGS. 4 and 5 and other drawings whereappropriate. Processes in FIGS. 4 and 5 are performed, provided that theuser with the HMD 100 has a meal. The HMD 100 images a meal in front ofthe user so as to specify contents of the meal (ingredients) from theobtained image and displays calories and constituents of the ingredientsin the display device 22. Therefore, the user may have a meal whilechecking the calories and the constituents. The HMD 100 is also capableof displaying a person who is having a meal in a remote location in thedisplay device 22. By this, the user may virtually have a meal with theperson in the remote location. Meanwhile, the user may get tired whenmasticating (chewing) ingredients during the meal. However, the user maynot be aware of fatigue if the user wears the HMD 100, and therefore,the accumulated fatigue may harm a user's body. Accordingly, thisprocess notifies the user of fatigue of the user during a meal andperforms display in accordance with the fatigue so that usability of theHMD 100 by the user is improved.

The process in FIG. 4 is started when the user inputs informationindicating that the user starts having a meal using the input device 20or when the control device 10 determines that the user has startedhaving a meal based on an image captured by the camera 16. At a timepoint when the process in FIG. 4 is started, a fatigue point, describedbelow, has been reset (a fatigue point=0).

When the process in FIG. 4 is started, first, the acceleration obtainingunit 50 obtains a value of measurement performed by the accelerationsensor 12 in step S10. The acceleration obtaining unit 50 transmits theobtained measurement value to the mastication determination unit 52. Instep S12, the mastication determination unit 52 determines whetheracceleration is equal to or larger than a predetermined value (α). Here,“α” is a median of a range of acceleration which is possibly detectedwhen the user moves a mouth. Specifically, when the acceleration isequal to or larger than the predetermined value α, it is highly likelythat the user is moving the mouth. The acceleration is approximately 0if the user does not move the mouth at all. Therefore, amplitude of theacceleration indicates a maximum value of the acceleration in one motionof opening and closing of the mouth.

The process in step S10 is repeatedly performed while the determinationin step S12 is negative. However, when the determination is affirmativein step S12, the process proceeds to step S14.

In step S14, the mastication determination unit 52 determines that themouth is moving. In step S16, the mastication determination unit 52determines whether the motion of the mouth has been stopped.Specifically, the mastication determination unit 52 determines whetheramplitude of the acceleration is stable in the vicinity of 0. When thedetermination is negative in step S16, the process proceeds to step S18where the mastication determination unit 52 obtains a measurement valueof the acceleration sensor 12 through the acceleration obtaining unit 50again. Thereafter, the process returns to step S16.

On the other hand, when the determination is affirmative in step S16,the process proceeds to step S20. Here, in FIG. 6A, a waveform of theacceleration obtained when the user is chewing (masticating) a food isillustrated. In FIG. 6B, a waveform of acceleration obtained when theuser is speaking (making a speech) is illustrated. As is apparent fromcomparison between FIGS. 6A and 6B, amplitude waveforms in which theacceleration is continuously changed in accordance with a load obtainedwhen upper and lower back teeth repeatedly abut on each other areobtained while the user is performing chewing. Specifically, when theuser is performing chewing, heavy loads are applied to the back teethwhich are indicated by pulses of large amplitude as illustrated in FIG.6A. On the other hand, an applied load is negligible when the user isspeaking, and therefore, small amplitude and a long cycle are attainedas illustrated in FIG. 6B.

In step S20, the mastication determination unit 52 determines whether avalue equal to or larger than γ is periodically obtained in a unit oftime. Here, a value of the acceleration obtained while the user isspeaking does not exceed “γ”. However, a value of the accelerationobtained while the user is performing chewing frequently exceeds “γ”.

When the determination is negative in step S20, the process proceeds tostep S22 where the mastication determination unit 52 determines that theuser is speaking. Specifically, when the waveform of the accelerationillustrated in FIG. 6B is obtained, the mastication determination unit52 determines that the user is speaking. Thereafter, the processproceeds to step S28.

On the other hand, when the determination is affirmative in step S20,the process proceeds to step S24 where the mastication determinationunit 52 determines that the user is chewing a food. Specifically, whenthe waveform of the acceleration illustrated in FIG. 6A is obtained, themastication determination unit 52 determines that the user is performingchewing. Thereafter, the process proceeds to step S26 where the fatiguedegree calculation unit 54 executes a fatigue determination process, andthereafter, the process proceeds to step S28.

Here, the fatigue determination process in step S26 will be described indetail with reference to the flowchart of FIG. 5. In the process in stepS26, it is determined whether the user is chewing a hard food or a softfood. When it is determined that the user is chewing a hard hood, it isestimated that the user gets tired and a fatigue point of the user isincremented.

In the process of FIG. 5, first, in step S50, the fatigue degreecalculation unit 54 measures a period of time (a chewing time) T from astart of chewing (a start of a meal) to an end of chewing (an end of themeal) and an amplitude A of acceleration in a range of the chewing timeT. In this case, the fatigue degree calculation unit 54 obtains anacceleration waveform from the start of chewing to the end of chewing asillustrated in FIGS. 7A and 7B, and a chewing time T is obtained inaccordance with the waveform. Then the fatigue degree calculation unit54 measures amplitudes A of the acceleration waveform in a range of thechewing time T.

In step S52, the fatigue degree calculation unit 54 determines whetherthe amplitude A of the acceleration in the start of chewing in the rangeof the chewing time T is equal to or larger than a predetermined value Sor smaller than the predetermined value S. The amplitude A is equal toor larger than the predetermined value S in the case of FIG. 7A, whereasthe amplitude A is smaller than the predetermined value S in the case ofFIG. 7B. The predetermined value S is a smallest value in accelerationamplitude obtained when the user starts chewing a hard food.

In step S54, the fatigue degree calculation unit 54 determines a periodof time in which the acceleration amplitude at the start of chewing inthe range of the chewing time T is reduced to α % of the amplitude A atthe time of the start of chewing (a period of time until theacceleration amplitude becomes smaller than (α/100)×A) as Tf and aperiod of time from when the amplitude is reduced to α % of theamplitude A to the end of chewing as Tr (refer to FIGS. 7A and 7B).

In step S56, the fatigue degree calculation unit 54 determines whetherthe amplitude A at the start of chewing is equal to or larger than thepredetermined value S and a value of Tr/T is equal to or larger than β %(β is 30, for example). As the user chews a harder food, the userintends to continue chewing for a longer time even after the foodbecomes soft, and therefore, a larger value of Tr/T means a harder food.

Accordingly, when the determination is negative in step S56 (in the caseof the waveform in FIG. 7B), it is estimated that the user is not tiredsince the user is chewing a soft food (such as rice or tofu), andtherefore, a fatigue point is not incremented, the process in FIG. 5 isterminated, and the process proceeds to step S28 of FIG. 4.

On the other hand, since the case where the determination is affirmativein step S56 (in the case of the waveform of FIG. 7A) means that the useris chewing a hard food (such as meat or a rice cake), it is determinedthat the user is tired since the user eats a chewy food and the fatiguedegree calculation unit 54 increments a fatigue point in step S58.Specifically, the fatigue degree calculation unit 54 increments thefatigue point accumulated from the start of the meal by one. After theprocess in step S58, the process proceeds to step S28 of FIG. 4.

Referring back to FIG. 4, when the process proceeds to step S28, thedisplay controller 56 determines whether the fatigue point exceeds apredetermined value. In step S28, the display controller 56 determineswhether the user gets tired in the meal in a degree equal to or largerthan a predetermined value. When the determination is negative in stepS28, the process returns to step S10, and otherwise, the processproceeds to step S30.

When the process proceeds to step S30, the display controller 56executes display control based on the fatigue point. For example, thedisplay controller 56 displays information on fatigue in the displaydevice 22 based on the fatigue point. For example, the displaycontroller 56 displays information on a notification indicating that theuser is tired and information indicating an action to be performed toalleviate the fatigue. In this case, the display controller 56 maydisplay information including at least one of text and an image as theinformation on the fatigue. Alternatively, the display controller 56 maychange a display form of characters or images (such as illustrations)based on the fatigue point. For example, when the fatigue point issmaller than a predetermined value, a small character size is used orcharacters are displayed in color which is less significant (quietcolor) whereas when the fatigue point is larger than the predeterminedvalue, a large character size is used or characters are displayed insignificant color (eye-catching color). Alternatively, the display formmay be changed in accordance with the fatigue point by another method.Here, the display controller 56 may display the information on thefatigue when a chewing action is not performed (when acceleration is notdetected). For example, the display controller 56 may display theinformation on the fatigue at a timing immediately after the chewingaction ends.

The information on the fatigue may be output by audio through the audiooutput device 24 instead of the display in the display device 22. Inthis case, the display in the display device 22 and the output throughthe audio output device 24 may be simultaneously performed.

The display controller 56 may differentiate the display form of theinformation to be displayed in the display device 22 (information oncalories and information on ingredients, for example) based on thefatigue point, in addition to the display of the information on thefatigue. For example, when the fatigue point is large, large charactersmay be used to indicate the information displayed in the display device22 or an eye-catching color may be used to indicate the informationdisplayed in the display device 22 so that the information is easilyrecognized. By this, even if the user is tired, the user may not missthe information displayed in the display device 22. Furthermore, thetired user may not be further tired when viewing the informationdisplayed in the display device 22.

When the process in step S30 is terminated as described above, theprocess returns to step S10 and the process described above isrepeatedly performed. When the user inputs the end of the meal using theinput device 20 or when the control device 10 determines the end of themeal based on an image captured by the camera 16, all the processes inFIGS. 4 and 5 are terminated.

As is apparent from the description above, according to the firstembodiment, the acceleration obtaining unit 50, the masticationdetermination unit 52, and the fatigue degree calculation unit 54realize a function of a detection unit which detects a degree of fatigueof the user with the HMD 100.

As described above in detail, according to the first embodiment, thecontrol device 10 includes the acceleration obtaining unit 50 whichobtains a measurement value of the acceleration sensor 12 included inthe HMD 100 and the mastication determination unit 52 which determineswhether the user is chewing or the user is speaking based on ameasurement value obtained by the acceleration sensor 12. By this, thecontrol device 10 may easily determine whether the user is chewing afood or the user is speaking with high accuracy based on theacceleration obtained when the user moves the mouth without using animage of the mouth, for example.

According to the first embodiment, the mastication determination unit 52performs the determination based on the amplitude of the measurementvalue obtained by the acceleration sensor 12. By this, the determinationmay be appropriately made taking the different acceleration waveforms atthe time of chewing and at the time of speaking (refer to FIGS. 6A and6B) into consideration.

According to the first embodiment, the fatigue degree calculation unit54 calculates the fatigue degree (the fatigue point) of the user basedon the result of the determination performed by the masticationdetermination unit 52. By this, the degree of the fatigue of the usermay be easily calculated.

In the first embodiment, when the mastication determination unit 52determines that the user is chewing a food (S24), the fatigue degreecalculation unit 54 calculates a fatigue degree of the user based onamplitude of the acceleration waveform (S26). By this, the determinationas to whether the user gets tired due to the chewing may be accuratelymade. In this case, the fatigue degree of the user is calculated bydetermining whether the user is chewing a hard food or a soft food inaccordance with the amplitude of the acceleration waveform, andtherefore, the fatigue degree of the user may be accurately determinedbased on characteristics of the chewing action of the user.

In the first embodiment, the display of the display device 22 iscontrolled based on the fatigue point calculated by the fatigue degreecalculation unit 54. By this, a notification indicating the fatigue orinformation for alleviating the fatigue may be transmitted to the user.Furthermore, the display device 22 may perform conspicuous display inaccordance with the fatigue degree of the user. Accordingly, the usermay improve the usability of the HMD 100 during a meal.

Although the case where the fatigue point is incremented only when ahard food is chewed is illustrated in the first embodiment, the presentdisclosure is not limited to this. For example, a predetermined numberof points (points smaller than the case where a hard food is chewed) maybe incremented even when a soft food is chewed. Alternatively, thenumber of points to be incremented may be differentiated in accordancewith the chewing time T.

Second Embodiment

Next, a head mounted display 100 according to a second embodiment willbe described in detail with reference to FIGS. 8 to 10. The secondembodiment is characterized in that a fatigue degree (a fatigue point)of a user is calculated using an image captured by a camera 16 inaddition to a measurement value of an acceleration sensor 12.

FIG. 8 is a functional block diagram illustrating a control device 10′of a head mounted display 100 according to the second embodiment. Asillustrated in FIG. 8, in addition to the functions of the unitsrealized in the control device 10 of the first embodiment, a function ofan image obtaining unit 60 is realized in the control device 10′according to the second embodiment. The image obtaining unit 60 obtainsan image captured by the camera 16 and transmits the image to a fatiguedegree calculation unit 54. The fatigue degree calculation unit 54calculates a fatigue point of the user based on a fatigue determinationtable 62 (refer to FIG. 10).

Next, the process executed by the control device 10′ of the secondembodiment will be described with reference to a flowchart of FIG. 9. Inthe flowchart of FIG. 9, portions different from the flowchart of FIG. 4are denoted by thick line frames.

In the process in FIG. 9, after the process in step S10 is executed, theimage obtaining unit 60 obtains the image captured by the camera 16 instep S11. The order of step S10 and step S11 may be reversed or theprocesses in step S10 and step S11 may be simultaneously performed.

Thereafter, a process from step S12 to step S24 is performed similarlyto the first embodiment. When a mastication determination unit 52determines that the user is chewing a food in step S24, the processproceeds to step S26′ where a process of incrementing a fatigue point isperformed based on a chewing time, the food, and a dish with referenceto the fatigue determination table 62.

Here, the fatigue determination table 62 has a configuration asillustrated in FIG. 10. Specifically, the fatigue determination table 62includes items of a “food”, “chewability”, a “cutlery”, “usability”, anda “fatigue determination time”. In the fatigue determination table 62,chewability of each food and usability of cutlery are defined.Furthermore, in the fatigue determination table 62, a food, a cutlery,and a period of time before it is determined that the user gets tired(the fatigue determination time) when the user eats a certain food aredefined. For example, in a case where the user eats meat using a knifeand fork, if a chewing time exceeds Td, a fatigue point is incrementedby 1. The magnitude relationships among the times illustrated in FIG. 10is represented as follows, for example: Td<Th<Tm<Tl<Te. In the fatiguedetermination table 62, the items of “chewability” and “usability may beomitted.

The fatigue degree calculation unit 54 obtains an image (an imageobtained immediately before the process in step S14 is performed, forexample) obtained before it is determined that a mouth is moved (beforeit is determined that the determination in step S12 is affirmative). Thefatigue degree calculation unit 54 determines a food that the user eatsand a cutlery used by the user by performing image processing, such astemplate matching, on the obtained image. Furthermore, the fatiguedegree calculation unit 54 obtains a period of time from when it isdetermined that the mouth is moved (S14) to when it is determined thatthe movement of the mouth is stopped (S16) (a chewing time T).

Then the fatigue degree calculation unit 54 extracts a fatiguedetermination time corresponding to the information (on the food and thecutlery) obtained by the image processing from the fatigue determinationtable 62, compares the extracted fatigue determination time with thechewing time T, and increments a fatigue point if the chewing time T islonger.

Thereafter, a process in step S28 and step S30 is the same as that inthe first embodiment described above. Specifically, also in the secondembodiment, a display controller 56 controls display of a display device22 in accordance with the fatigue point similarly to the firstembodiment.

As is apparent from the description above, according to the secondembodiment, an acceleration obtaining unit 50, the image obtaining unit60, the mastication determination unit 52, and the fatigue degreecalculation unit 54 realize a function of a detection unit which detectsa degree of fatigue of the user with the HMD 100.

As described above, according to the second embodiment, the fatiguedegree calculation unit 54 calculates a fatigue point based on an imageof a food and cutlery captured by the camera 16 and acceleration, andtherefore, the fatigue point of the user may be calculated with highaccuracy. Since the display of the display device 22 is controlled basedon the fatigue point calculated with high accuracy in the secondembodiment, the display may be performed taking the fatigue of the userinto consideration. Accordingly, the user may improve the usability ofthe HMD 100.

Although the case where the fatigue point is calculated usinginformation on a food and information on cutlery obtained from the imageis described, the present disclosure is not limited to this, and thefatigue point may be calculated using one of the information on a foodand the information on cutlery.

According to the second embodiment, the fatigue determination table 62may be provided for each user (in accordance with a user). For example,content of the fatigue determination table 62 may be changed dependingon likes and dislikes of the user, such as a user who is good at usingknife and fork and a user who is good at using chopsticks, since usersare a variety of ages and have a variety of dexterity.

In the foregoing embodiments, the gyroscope sensor 14 may be used tocalculate the fatigue point. For example, when the user views charactersand the like displayed in the display device 22 (characters and the likedisplayed in a lower portion in a screen of the display device 22), ahead may direct downward, and therefore, the user may get more and moretired every time the user views the screen. In this case, if a result ofdetection of the gyroscope sensor 14 indicates a predetermined value, itmay be determined that the user views the lower portion in the screen,and a fatigue point may be incremented based on the number of times theuser views the lower portion. By this, the user may calculate thefatigue point based on the number of times the user views the displaydevice 22. Also in this case, a fatigue point to be incremented may bedifferentiated in accordance with attributes of the user (an age, agender, a degree of disability).

In the foregoing embodiments, it may be determined whether the user ischewing a food or the user is speaking based on audio information inputthrough the microphone 18.

In the foregoing embodiments, a determination criterion or a value of afatigue point to be incremented may be changed depending on an age, agender, a type (a healthy person, an elderly person, or ahearing-impaired person) or the like. By this, a fatigue point may becalculated and the display device 22 may be controlled taking theattributes of the user into consideration.

Although the case where the display device 22 is controlled based on thefatigue when a food is chewed is illustrated in the foregoingembodiments, the present disclosure is not limited to this. For example,a fatigue point may be incremented based on eyestrain and the displaydevice 22 may be controlled based on the fatigue point.

Although the case where display of the display device 22 is controlledbased on the fatigue is illustrated in the foregoing embodiments, thepresent disclosure is not limited to this. Specifically, the fatiguedegree calculated by the fatigue degree calculation unit 54 may be usedto improve usability of the HMD 100 instead of the control of thedisplay device 22.

The processing functions described above may be realized by a computer.In this case, programs in which content of processes of functions to beincluded in the processing apparatus are provided. The processingfunctions are realized in the computer by executing the programs in thecomputer. The programs including the processing content describedtherein may be recorded in a computer-readable recording medium (exceptfor carrier waves).

When the programs are to be distributed, a portable recording medium,such as a digital versatile disc (DVD) and a compact disk read onlymemory (CD-ROM) which include the programs recorded therein, is sold,for example. Alternatively, the programs may be stored in a storagedevice of a server computer and transmitted from the server computer toanother computer through a network so that the program is distributed.

The computer which executes the programs stores the programs recorded inthe portable recording medium or the programs transmitted from theserver computer in a storage device thereof, for example. Then thecomputer reads the programs from the own storage device and executesprocesses in accordance with the programs. The computer may directlyread the programs from the portable recording medium and executeprocesses in accordance with the programs. Alternatively, the computermay execute a program based on the received program every time theprogram is transmitted from the server computer.

All examples and conditional language provided herein are intended forthe pedagogical purposes of aiding the reader in understanding theinvention and the concepts contributed by the inventor to further theart, and are not to be construed as limitations to such specificallyrecited examples and conditions, nor does the organization of suchexamples in the specification relate to a showing of the superiority andinferiority of the invention. Although one or more embodiments of thepresent invention have been described in detail, it should be understoodthat the various changes, substitutions, and alterations could be madehereto without departing from the spirit and scope of the invention.

What is claimed is:
 1. A non-transitory computer-readable storage mediumstoring a program that causes a processor of a head mounted display toexecute a process, the process comprising: obtaining a result ofmeasurement performed by an acceleration sensor included in the headmounted display; determining whether a user with the head mounteddisplay is chewing a food or the user is speaking based on the result ofthe measurement performed by the acceleration sensor; and calculating afatigue degree of the user based on a determination result obtained inthe determining process.
 2. The storage medium according to claim 1,wherein the determining includes determining whether the user is chewinga food or the user is speaking based on a waveform of the result of themeasurement performed by the acceleration sensor.
 3. The storage mediumaccording to claim 1, wherein the calculating includes calculating thefatigue degree of the user based on amplitude of a waveform of theresult of the measurement performed by the acceleration sensor when itis determined that the user is chewing a food.
 4. The storage mediumaccording to claim 1, wherein the process comprising obtaining an imagecaptured by an imaging unit included in the head mounted display,wherein the calculating includes calculating a fatigue degree of theuser based on the image.
 5. The storage medium according to claim 4,wherein the calculating includes: specifying at least one of informationon a food chewed by the user and information on cutlery used by the userusing the captured image; and calculating a fatigue degree of the userbased on the specified information.
 6. The storage medium according toclaim 1, wherein the calculating includes differentiating a calculationcriterion of the fatigue degree depending on a type of the user.
 7. Thestorage medium according to claim 1, wherein the process comprisingcontrolling a display included in the head mounted display based on thecalculated fatigue degree of the user.
 8. A head mounted display,comprising: an acceleration sensor; and a processor coupled to theacceleration sensor and configured to: obtain a result of measurementperformed by an acceleration sensor, determine whether a user with thehead mounted display is chewing a food or the user is speaking based onthe obtained result of the measurement, and calculate a fatigue degreeof the user based on a determination result obtained in the determiningprocess.
 9. A non-transitory computer-readable storage medium storing aprogram that causes a processor of a head mounted display to execute aprocess, the process comprising: detecting a fatigue degree of a userwith the head mounted display; controlling visibility of display in thehead mounted display based on the detected fatigue degree; obtaining aresult of measurement performed by an acceleration sensor included inthe head mounted display; and determining whether the user is chewing afood or the user is speaking based on the result of the measurementperformed by the acceleration sensor, wherein the detecting includescalculating a fatigue degree of the user based on a result of thedetermining.